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ABSTRACT  ' 

In  the  Secretary  of  Defense's  Annual  Report  for  FY'79  to  the 
Congress,  released  on  2 February  1978,  he  indicated  that  the  Depart-  j 

ment  of  Defense  (DOD)  was  currently  testing  a new  concept  for  con-  ! 

tractor  source  selection  on  major  programs  called  the  "four-step" 

process.  He  further  indicated  that  a decision  would  be  made  in  j; 

February  on  whether  DOD  would  adopt  this  process.  DOD's  en-  j; 

courage  ment  of  the  publication  by  the  Wall  Street  Journal  of  a ^ i] 

favorable  article  on  the  subject  on  6 March  would  seem  to  indicate  i 

that  this  decision  has  been  made  and  that  DOD  intends  to  go  ahead 
and  adopt  the  concept  as  a matter  of  policy. 

This  paper  questions  both  the  desirability  and  the  necessity 
for  DOD  to  do  so,  since  the  objectives  of  "fovir-step"  can  be 
accomplished  in  other  ways.  And  these  ways  do  not  risk  reducing 
program  results,  as  "four-step"  does. 
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THE  PENTAGON  "FOUR -STEP" 


The  Pentagon  is  dancing.  Although  it  has  one  of  the  best 
records  in  Government  in  the  area  of  controlling  cost  growth^ 
in  spite  of  being  involved  in  perhaps  the  most  complex  management 
activity  in  which  our  Society  engages  --  acqxiiring,  over  lengthy 
time-spans,  the  advanced  technological  material  that  provides  for 
our  national  security  --  it  nevertheless  feels  driven  politically  to 
offer  still  another  method  for  "discouraging  cost  over-runs.  " This 
is  its  new  procedure  for  choosing  the  private  organizations  who  will 
develop  and  produce  its  major  military  systems,  called  "four -step 
source  selection.  " And  so,  by  means  of  appropriate  interviews, 
the  new  procedure  is  announced  publicly  in  a respectable  medium 
like  the  Wall  Street  Journal  of  March  6,  1978  (see  Appendix).  And 
its  validity  is  attested  to  by  solemnly  calling  attention  in  such  inter- 
views to  the  two-year  test  that  was  run  on  it  "involving  66  bids  on  17 
projects.  " 


Further,  perhaps  because  no  claim  is  being  made  that  the 

cost  of  the  highly  expensive  defense  programs  to  which  "four-step" 

fzl 

is  to  be  applied  is  going  to  be  reduced  by  the  new  procedure*^  ^ -- 


The  Armed  Forces  Journal,  March  1978,  estimates,  based  on 
GAO  data,  that  DOD's  weapon  cost  growth  averages  "less  than 
half  of  other  major  federal  government  acquisitions.  " 

[2] 

In  fact,  the  Department  of  Defense's  (DOD's)  Director  of  System 
Acquisition  is  quoted  qxiite  openly  as  saying  "This  won't  reduce 
the  cost  of  weapons;  in  fact,  at  the  outset,  the  price  may  seem 
higher.  ..." 
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only  the  difference  between  the  estimated  cost  of  the  programs 
and  their  actual  cost  --  a claimed  finding  from  the  tests  of 
seemingly  greater  significance  is  made  about  "four-step"  in  the 
public  announcement:  that  it  "produce(8)  technical  proposals 
that  are  more  thoughtful  and  imaginative  than  under  the  current 
system.  " Finally,  the  claim  is  made  that  the  new  technique 
will  prevent  or  reduce  certain  "abuses"  of  the  former  approach 
to  source  selection  called  "technical  leveling",  "technical  trans- 
fusion", "auctioning"  (through  the  use  of  the  "best  and  final  offer" 
technique),  and  "buying  in"  --  all  of  which  will  increase  the 
fairness  with  which  the  source  selection  process  is  conducted. 
Measuring  the  degree  to  which  "four-step"  would  eliminate  or 
reduce  these  four  "abuses"  was,  in  fact,  the  stated  official  ob- 
jective of  the  test. 

Perhaps  so.  Maybe  the  test  results  do  substantiate  all  of 
tliese  claims.  But  there  is  room  for  doubt  about  this  --  as  well 
as  whether  a controlled  test  of  any  type  was  actually  ever  run  and 
whether  the  right  things  were  tested. 

So  what?  What  if  the  new  approach  doesn’t  turn  out  to  be 
any  better  than  the  old?  After  all,  it  can  always  be  argued  that  it 
doesn't  matter  that  there  is  a new  set  of  rules  in  DOD  for  selecting 
sources,  because,  in  a competitive  environment,  businessmen  can 

I 

always  do  as  well  \inder  one  set  of  rules  as  another  --  provided, 
of  course,  that  the  rules  are  known  and  understandable  by  all  and  are 
consistently  applied.  And  these  the  DOD  always  does  a good  Job  of 
assuring.  So  why  be  concerned? 
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The  reason  for  the  concern  is  that  the  new  approach  may  be 
worse  than  the  old,  in  a vital  respect  --  a respect  that  has  not 
even  been  tested,  much  less  tested  successfully  And  that  is,  it 
may  lead  to  worse  program  results.  And  program  results  are 
"the  name -of -the -game"  in  systems  acquisition,  not  better  proposals. 

Admittedly,  to  obtain  data  on  comparative  results  in  programs 
that  take  as  much  as  15  years  to  accomplish  would  take  much  time. 
And  the  so-called  "abuses"  need  to  be  alleviated  now,  as  well  as 
action  taken  on  the  cost-growth  front.  Failure  to  do  this  latter,  in 
fact,  could  threaten  what  may  already  be  a too-low  defense  budget. 

But  neither  should  be  dealt  with  via  "four-step",  because  it  is  the 
contention  of  this  paper  that: 

(1)  We  can  predict  diminished  program  results  from 
"four-step",  both  in  performance  and  costliness, 
from  what  we  already  know  --  to  the  detriment  of 
both  the  Government  and  its  suppliers. 

(2)  Maximizing  program  results,  in  terms  of  getting  the 
right  materiel  into  the  hands  of  fighting  forces  in 
time  for  it  to  have  the  greatest  first-line  operational 
life,  is  the  primary  job  of  defense  acquisition  officials; 
providing  for  equity  in  the  process  of  selecting  sources 
to  provide  such  materiel  is  important,  but  secondary. 
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(3)  While  correcting  the  so-called  "abuses"  that  are  taking 
place  in  today's  source  selections  is  necessary,  each 
can  be  corrected  by  a means  other  than  one  like  "four- 
step"  which  risks  the  accomplishment  of  the  primary 
job.  They  can,  in  fact,  be  dealt  with  directly. 

Each  of  these  contentions  will  be  discussed  in  turn.  If  they  are 
valid,  "four -step"  should  be  dropped,  the  "abuses"  addressed  directly, 
and  an  approach  taken  to  major  source  selections  which  recognizes 
that  they  are  not  isolated  activities  in  themselves  to  which  a uniform 
set  of  rules  can  be  applied.  Rather,  they  are  only  steps  in  a long  chain 
of  events  through  which  a specific  national  security  capability  is  ac- 
quired. As  such,  they  each  need  to  be  shaped  and  conducted,  like  any 
other  part  of  the  design  process,  to  meet  the  needs  of  the  strategy  de- 
vised for  acquiring  the  particular  system  at  issue.  And  this  means 
taking  into  account  in  a maximum  way  such  parameters  --  to  name 
just  a few  --  as  a program's  priorities,  a program's  uncertainties, 
and  the  intended  relationship  between  the  Government  and  the  winning 
contractor (s ) after  the  source  selection  has  been  completed.  Procure- 
ment is  not  an  end  in  itself,  much  less  its  contracting  aspects. 


For  more  on  this  subject,  see  Procurement  and  the  Systems 
Engineer.  N.  Waks,  MITRE  MTP-1,  May  1965. 
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DIMINISHED  RESULTS  FROM  "FOUR -STEP" 

The  predictably  diminished  program  results  in  terms  of 
performance  and  costliness  both  spring  from  the  same  source.  And 
that  is  the  requirement  under  "four-step"#  as  an  admittedly  basic 
difference  between  it  and  the  former  procedure,  of  strictly  limited 
communication  between  the  buyer  (the  Government)  and  the  potential 
sellers  (the  industrial  bidders)  during  the  proceeding,  in  order  to 
help  assure  the  equitableness  of  the  selection.  Yet  there  should  be 
maximum  communication  during  this  period,  in  order  to  assure  that 
what  the  respective  bidders  are  offering  will  in  fact  satisfy  the  Gov- 
ernment's needs  and  preferences.  That  is,  given  the  many  uncer- 
tainties that  ordinarily  exist  at  this  "handover"  point  in  an  advanced 
technological  program  --  even  great  uncertainty  at  times  about 
whether  the  program  can  be  accomplished  in  a timely  manner  at  all  — 
it  is  absolutely  mandatory  that  not  only  maximum  clarifying  discussion 
of  the  need  and  offers  take  place,  which  "four-step"  still  permits  to  a 
degree,  but,  since  the  offers  are  really  proposed  "solutions"  to  the 
Government's  "problem",  both  maximum  exploration  of  the  validity  of 
each  and  the  relative  degree  of  fit  between  the  two. 

Indeed,  the  wecikness  in  even  the  current  source  selection 
technique,  in  which  unlimited  discussion^  permitted,  is  that  the 
Government  does  not  know  how  to  accomplish  this  vital  goal  of  matching 
problem  and  solution  well  enough  at  this  critical  stage  of  a program  so 
as  to  avoid  later  major  program  difficulties  directly  attributable  to 
failures  of  communication  at  source  selection  time.  And  this  has  led. 
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at  times,  to  the  most  costly  programs  of  all:  those  which  meet  their 
development  contracts  but  which  are  never  produced  or  deployed 
because  they  insufficiently  met  the  military  need  which  the  program 
was  initiated  to  satisfy.  Can  there  be  any  doubt,  therefore,  that  a 
source  selection  procedure  for  major  defense  programs  which  delib- 
erately goes  in  the  direction  of  reducing  the  amount  and  type  of  dis- 
cussion that  government  reviewers  of  proposals  can  initiate  when  they 
encounter  questionable  aspects  of  the  offers  they  receive  will  cause 
the  cost  of  DOD's  overall  defense  acquisition  program  to  go  up  --  and 
go  up  not  only  in  dollars  but  in  force  readiness? 

A good  source  selection  proceeding  on  a major  program,  as 
those  who  have  participated  in  one  are  well  aware,  involves  of  necessity 
a good  deal  of  iteration  between  the  potential  contracting  parties,  as 
both  the  Government  statement  of  need  gets  shaped  and  refined  as  a 
result  of  what  is  offered,  and  the  Government,  in  turn,  not  only  com- 
paratively evaluates  bidder  proposals  but  attempts  to  validate  them 
(prevent  "competitions  of  exaggeration"  by  verifying  that  what  bidders 
promise  can  in  fact  be  done  and  be  done  in  the  time  and  for  the  cost 
estimated).  As  a result,  it  is  quite  predictable  that  if  such  complete, 
unrestrained  iteration  does  not  take  place  during  the  source  selection 
with  at  least  some  of  the  bidders  in  the  competitive  range,  then  it 
most  assuredly  will  have  to  take  place,  in  one  form  or  another,  at  a 
later  date,  probably  at  various  Government  review  points  in  the  pro- 
gram. The  fact  cannot  be  avoided  that,  at  some  point,  there  must  be 
a full  meeting  of  the  minds  about  the  degree  of  appropriateness  of  the 
approach  that  is  being  taken  to  meeting  the  need.  And  until  it  is,  time 
and  dollars  will  be  flowing,  much  of  which  could  be  wasted.  Thus, 
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it  can  be  argued  that  even  the  equity  goals  which  are  being  used  to 
justify  "four-step",  cannot  in  fact  be  achieved  under  it.  An  attempt 
will  always  be  made  eventually  to  change  the  winning  proposal  at 
least  to  whatever  degree  is  necessary. 

For  those  who  think  that  the  "four-step"  mandate  of  limited  dis- 
cussion of  bidder  deficiencies  before  contractor  selection  will  always 
be  aborted  during  the  final  negotiations  with  this  winning  bidder, 
rather  than  being  deferred,  a warning  is  in  order.  And  that  is  that 
the  very  advocates  in  industry  and  Government  who  caused  "four-step" 
to  come  about  in  the  first  place  will  be  giving  maximum  attention  to 
this  very  obvious  possibility.  That  is.  Company  A is  most  assuredly 
not  going  to  let  itself  be  eliminated  from  the  race  with  Company  B 
for  a multi-million  dollar  prize,  under  one  set  of  rules,  and  then 
stand  by  while  "B"  's  proposal  is  immediately  fixed  up  under  another. 
Indeed,  there  is  every  reason  to  believe  that  Government  evaluators 
will  lean  over  backwards  not  to  fix  up  "B"  's  proposal  during  these 
final  negotiations,  in  order  not  to  put  themselves  in  a position  to  be 
criticized  on  this  score  while  the  glare  of  the  spotlight  of  a formal 
source  selection  still  exists  --  thus  leading  to  even  less-than- 
permissible  rectification  of  the  winning  proposal.  And  their  tendency 
to  be  over-cautious  at  this  stage  will  be  added  to  by  the  earlier  diffi- 
culty the  test  results  show  they  have  in  distinguishing  at  proposal  sub- 
mission stages  just  what  ^ permissible  discussion  of  these  proposals 
under  "four-step"  and  what  is  not.  The  limited-communication  man- 
date of  "four -step",  that  is,  is  likely  to  be  observed  in  practice,  for 
several  reasons,  to  a degree  greater  than  required  even  by  policy  -- 
to  the  detriment  of  both  the  field  commander  with  a weapon  that  doesn't 
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do  the  job,  or  is  late,  and  the  industrialist  with  the  "successful" 
development  contract  who  receives  no  follow-on  production  order, 
or  receives  one  which  lasts  for  only  a few  years  before  militarily 
obsolescing. 
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THE  PRIMARY  JOB  OF  DEFENSE  ACQUISITION  OFFICIALS 


It  is  a basic  tenet  of  a political  democracy  like  ours  that 
everybody  will  be  treated  fairly  by  the  Government.  Providing  for 
equality-of -opportxinity  in  the  provision  of  the  now  over  $80  billion 
a year  worth  of  goods  and  services  the  Government  purchases  each 
year  has,  in  fact,  been  a major  goal  of  Government  procurement 
policy  and  practices  since  at  least  the  Civil  War. 


And  this  has  not  been  simply  a matter  of  political  philosophy. 

Tlie  econoniic  success  of  our  capitalistic  Society  is  felt  to  depend 

heavily  on  pitting  private  organizations  against  eacli  other  in  free 

and  open  competitions  for  prizes  they  value  --  wliether  these  be 

consumer  and  industrial  markets  or  Government  contracts.  And 

wliile  some  of  the  concepts  about  free  private  enterprise  that  abound 

f4| 

are  a bit  simplistic  for  today's  complex  society  , their  stress  on 

tlie  benefits  that  derive  from  competition  --  competition  in  all  its 

various  forms  --  is  essentially  valid.  Harnessing  the  force  of 

individual  people  or  enterprises  acting  in  their  own  behalf  is  a pow- 

[51 

erful  form  of  governance 


f4] 
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Cf.  What  is  Private  Enterprise  Today?  , MITRE  MTP-145, 

N.  Wake,  December  1971. 

Cf.  The  Public  Use  of  Private  Interest,  The  Brookings  Institvition, 
C.  L.  Schultze,  currently  Chairman  of  the  Council  of  Elconomic 
Advisors,  1977. 
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Maximizing  the  equality-of-opportunity  criterion  in  Government 
procurement  to  take  advantage  of  these  self-interest  competitive  forces 
works  well  in  terms  of  program  results  and  costliness  when  the  Govern- 
ment knows  and  can  describe  exactly  what  it  wants,  as  in  the  ordinary 
case  of  typewriters  or  articles  of  clothing  for  the  Armed  Forces. 

It  works  less  well,  but  still  sufficiently  satisfactorily  for  most 
purposes  (if  supplemented  by  a little  common  sense  about  the  degree  of  ‘ 

"negotiation"  needed  in  a particular  case)  when  the  Government  knows, 
and  can  at  least  set  forth  in  physical  or  chemical  terms,  what  it  is 
that  what  it  is  procuring  should  do  --  sufficiently  so  that  it  can  objec- 
tively measure  and  then  compare  the  results  of  what  is  being  offered. 

A device  that  can  send  a radio  signal  a given  distance  might  be  an  example  ^ 

here. 

I 

However,  attempting  to  maximize  the  equality-of-opportunity  j 

criterion  contributes  little  to  making  a competition  effective  in  terms 
of  program  results: 

(a)  when  what  is  needed  cannot  be  adequately  described  by  the 
Government  in  terms  much  more  specific  than  a broad  tech- 
nical translation  of  a need  to  solve  some  military  problem; 

(b)  when  the  Government  has  no  way  of  objectively  meas-  ' 

uring  whether  what  is  being  offered  will  in  fact  solve 

this  military  problem;  and 
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(c)  when  the  materiel  item  being  obtained  is  being  acquired 
for  the  first  time  --  indeed  when  what  is  being  acquired 
is  really  not  materiel  at  all,  but  rather  is  merely  effort 
directed  towards  gaining  the  information  needed  to  help 
the  Government  decide  whether  it  can,  and  should  even  try, 
to  proceed  with  a technological  approach  to  solving  its 
military  problem; 

i.  e.,  in  sum,  when  what  is  being  acquired  is  a major  system  develop- 
ment effort.  In  fact,  the  equality-of-opportunity  criterion  becomes 
distinctly  secondary  in  such  cases  --  being  confined  to  considerations 
of  preserving  the  industrial  base  and  helping  to  assure  that  individual 
competitions  are  real.  And  yet  the  purpose  of  "four-step”  is  to  increase 
the  relative  importance  of  this  criterion  of  a good  source  selection,  and 
do  it  at  a time  when  the  growing  complexity  and  cost  of  major  defense 
systems  is  causing  the  balance  among  the  criteria  in  even  the  current 
approach  to  be  seriously  questioned  in  this  regard. 

Simply,  an  American  source  selection  activity  in  a major  de- 
fense program  has  always  been  a mixture  of  efforts  directly  towards 
picking  a source  in  the  most  equitable  manner  possible  at  the  same 
time  as  getting  the  most  promising  product  the  Government  can  for 
the  time  and  funds  available.  But,  by  the  mid- 5 Os,  the  concept  of 
full  and  free  competition  for  this  class  of  goods  had  begun  to  be  ser- 
iously questioned.  And  by  the  latter  half  of  the  60s,  it  was  openly 
recognized  in  the  Pentagon  that  these  twin  criteria  of  a good  source 
selection  activity  were  not  always  compatible  --  indeed,  that  they 
often  conflicted. 
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Specifically,  the  DOD  recognized  that  it  had  to  continue  to 
assure  that  it  always  had  available  an  industrial  base  of  suppliers 
able  to  undertake  such  a massive,  high-risk  enterprise  as  a major 
system  development,  and  that  the  long-range  competitive  atmosphere 
it  created  could  affect  this  base.  Further,  it  realized  that  it  had  to 
assure  in  each  formal  competition  that  a sufficient  number  of  such 
qualified  suppliers  became  interested  enough  in  the  competition  to 
make  the  competition  real,  a situation  which  could  be  affected  by  the 
competitive  atmosphere  surrounding  the  particular  competition. 
However,  importantly,  it  also  began  to  realize  that  it  did  not  need  to 
encourage  more  than  this  number  or  to  admit  into  the  arena  any  but 
the  most  qualified  bidders.  It  began  to  understand,  that  is,  that  bid 
lists  for  major  programs  had  to,  and  could,  be  quite  selective^^^. 

In  addition,  and  directly  to  the  point  here,  the  relentless  in- 
creases in  the  dollar  cost  and  lead-time  of  these  major  programs  led 
it  to  realize  that  getting  the  job  done  at  all  was  increasingly  in  doubt 
and  that,  as  a result  — to  state  it  in  the  extreme  --  the  equity  cri- 
terion was  becoming  a "luxury"  in  the  conduct  of  the  evaluative  phases 
of  a source  selection  also,  if  it  threatened  the  results  that  might  be 
expected  from  the  program. 


Cf.  Selective  Competition  in  New  Air  Weapon  Procurement, 
unpublished  Harvard  Business  School  doctoral  thesis, 

N.  Waks,  1961. 
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To  express  simply  what  was  driving  such  officials,  it  was 
realized  that  a $900  million  program  which  has  a three-year  first- 
line  operational  life  against  a particular  enemy  threat  is  a $300 
million  per  year  program.  But  it  becomes  a $450  million  per  year 
program  if  only  having  a two-year  life,  and,  of  course,  a $900 
million  a year  program  if  only  first-line  for  one  year.  And  to  the 
degree  that  it  is  less  than  fully  effective  when  it  gets  into  the  hands 
of  troops,  even  ahead  of  schedule,  program  cost-benefit  ratios 
begin  to  approach  infinity.  Thus  the  driving  thrust  of  major  source 
selections  increasingly  became,  and  has  remained  since  then,  an 
intimate  and  wide-ranging  iteration  between  the  Government  and  the 
Country's  most  qualified  and  available  suppliers  to  assure: 


(a) 

(b) 

(c) 

(d) 


that  such  suppliers  can,  in  fact,  provide  what  is 
being  described  in  the  bid  specifications, 
that  what  is  being  specified  will  in  fact  do  the 
military  job  needed, 

that  the  risks  involved  in  the  program  are  being 
overtly  identified  and  hedged,  and 
that  the  need  which  gives  rise  to  a program  in  the 
first  place  will  be  tracked  and  up-dated  in  whatever 
manner  is  necessary  to  assure  that  suppliers  will 
not  be  later  left  in  the  position  of  having  completed 
highly  successful  development  efforts  on  programs 
that  are  either  not  produced  or  deployed. 
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In  this  last  regard,  the  great  importance  that  defense  suppliers 
currently  attach  to  not  being  asked  to  go  into  full-scale  develop- 
ment on  programs  which  are  not  assured  of  being  produced  and 
deployed  if  successful  in  test  was  amply  illustrated  at  the  San  Diego 
Defense  Science  Board  meeting  on  systems  acquisition  this  past 
summer.  They  realize  that  success  in  being  selected  to  do  such 
development  is  a hollow  victory  if  not  accompanied  by  steps  that 
assure  the  success  of  the  program  as  well. 

Today  we  have  even  less  choice  about  the  primary  job  of 
defense  procurement  officials.  The  integ ratability  --  in  the  physical, 
functional,  and  schedule  sense  --  of  any  given  program  with  the 
other  programs  that  are  intended  to  help  satisfy  a mission  need  has 
become  a major  driver  of  the  interaction  activity  between  the  Gov- 
ernment and  its  suppliers'  system  engineers,  as  both  have  come  to 
realize  that  the  principal  goal  of  military  acquisition  is  to  help 
provide  greater  force  capabilities  rather  than  simply  to  add  superior 
items  of  materiel  to  the  military  inventory. 

Too,  the  relationship  between  the  interacting  parties  is,  of 
necessity,  becoming  less  and  less  at  arms-length  or  independent. 

The  job  is  becoming  so  difficult  that  Government  engineers  in  major 
programs  have  to  be  encouraged  in  all  ways  possible,  much  less 
permitted,  to  take  every  opportunity  to  discuss  bidder /contractor 
approaches  and  plans  for  programs,  lest  these  programs  wander 
down  the  costly  path  of  getting  started  wrong  and  staying  that  way. 
Recognition  of  this  over-riding  consideration  caused  Pentagon 
acquisition  executives  as  early  as  the  late  '60s  to  openly  seek  ways 
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in  a source  selection  of  separating  out  what  was  to  be  accomplished 
in  a particular  program  from  the  decision  of  who  would  provide  it. 

This  separating  of  the  "what"  from  the  "who"  is  not  an  easy  task;  but, 
as  the  new  (April  1976)  A-109  Government  policy  document  on  major 
system  acquisition  recognizes,  it  can  be  done  to  a great  degree,  if 
started  early  enough  in  the  system  cycle  so  that  a single  winner  is  not 
being  chosen  but  merely  the  most  qualified  contestants  for  a periodic- 
ally held  elimination  contest  in  which  selection  will  be  based  on  demon- 
strable results. 

The  blunt  question  must  thus  be  asked,  under  the  above  cir- 
cumstances, of  what  use  "four -step"  is  to  Government  acquisition 
officials  when  it  diverts  attention  and  effort  from  their  ever -more 
primary  job,  only  for  reasons  of  increasing  the  satisfaction  of  their 
secondary  one.  Indeed,  it  must  pointedly  be  asked,  "Is  it  really  even  to 
the  ultimate  advantage  of  the  intended  beneficiaries  --  defense 
suppliers  --  at  the  price  that  must  be  paid  for  it?  ". 
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TAKING  A DIRECT  APPROACH  TO  THE  "ABUSES" 


[71 

If  experienced  system  source  selection  people  know  all  this^  , 
why  then  the  movement  towards  trying  to  increase  the  equity  of  major 
source  selections  by  establishing  the  "four-step"  procedure?  Specif- 
ically, why  did  the  Pentagon  reply  with  a new  procedure  to  the  protests 
of  its  major  suppliers  that  source  selections  were  being  conducted  un- 
fairly as  a result  of  the  "buying  in",  "auctioning",  "technical  leveling", 
and  "technical  transfusions"  that  were  teiking  place? 

v_ 

The  answer  is  that  these  complaints  were  valid,  by  and  large, 
contributing  to  the  size  of  cost  "over-runs"  through  establishing 
initial  estimates  of  program  costs  that  everybody  knew  were  too  low 
right  from  the  beginning  and  creating  a source  selection  atmosphere 
that  was  fraught  with  the  opportunity  for  bidders  to  protest  officially 
to  the  General  Accounting  Office  and  to  higher  level  Defense  Department 
officials  that  the  results  of  various  competitions  should  be  over -turned 
as  a result  of  their  mishandling.  And  protest  they  did.  The  number  of 
protests  jumped  tremendously  in  the  '70s,  in  fact,  as  system  programs 
got  fewer  in  number  and  lasted  increasingly  longer,  making  the  stakes 
in  obtaining  any  given  one  go  up  precipitously  for  an  individual  defense 
firm.  In  turn,  these  mounting  protests  put  the  procurement  officials 
who  were  actually  conducting  the  source  selections  in  a "no-win"  position 
every  time  they  held  a competition.  They  were  criticized  to  higher 
level  officials,  that  is,  no  matter  how  good  a job  they  did  in  the  selection 
process. 


[7] 


Which  may  be  why  only  23%  of  the  Government  people  interviewed 
the  test  preferred  the  "four-step"  approach  for  general  use. 


in 


1 
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Thus  the  "abuses”  problem  had  to  be  fixed.  But  it  is  the 
contention  of  this  paper  that  it  did  not  have  to  be  fixed  by  adopting 
"four-step",  an  approach  which,  at  best,  gets  at  the  problem  at 
too  great  a price,  and  then  only  indirectly.  The  four  specific  concerns 
can  be  handled  directly,  as  the  following  indicates. 

(1)  In  a sense  two  of  the  concerns  "auctioning"  (through 
the  use  of  the  "best  and  final  offer"  technique)  and  "buying  in",  are 
simply  opposite  faces  of  the  same  problem  as  far  as  eliminating 
them  as  "abuses"  are  concerned.  In  the  former,  the  abuse  claimed 
is  that  the  Government  is  encouraging  bidders  to  bid  prices  below 
what  they  believe  warranted,  when  it  requires  them,  after  detailed 
and  repeated  discussions  of  proposal  deficiencies  and  parallel  contract 
negotiations  have  been  conducted,  to  make  a "best  and  final  offer". 

In  the  latter  ("buying  in"),  one  or  more  bidders  voluntarily  bid  prices 
which  they  know  to  be  below  the  likely  cost  of  the  program  to  them  -- 
knowing  full  well  that,  whether  or  not  the  procuring  official  suspects 
that  this  is  what  is  happening,  he  is  then  bound  to  make  the  award  to 
them  if  their  technical  proposals  are  at  least  acceptable.  This  is 
because  of  the  difficulty  of  explaining  any  other  action  in  the  face  of 
the  long-standing  Government  tradition,  bordering  on  a mandate,  of 
making  awards  to  the  lowest-priced  responsible  and  responsive  bidders 
whenever  possible. 
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But  these  two  "abuses"  do  not  call  for  "four-step". 

They  both  come  from  an  increasing  failure  of  Government  procure- 
ment officials  since  the  '50s  to  recognize  the  management  implications 
of  the  varying  relationships  the  Government  as  a buyer  can  establish 
with  its  suppliers  in  satisfying  some  need.  In  particular  here,  they 
follow  from  its  increasing  failure  to  recognize,  in  all  but  form,  that 
in  the  type  of  relationship  which  ordinarily  must  be  established  in  the 

class  of  procurements  to  which  "four-step"  is  being  applied  --  a 

[81 

Cost-type  relationship*^  --  the  notion  oK  'price"  does  not  exist  at  il, 

much  loss  the  concept  of  lowest  price.  In  this  relationship,  that  is, 
a contractor  is  not  an  arms -length  supplier  offering  to  do  a specific 
job  by  a given  date  for  a specific  price.  Rather,  he  is  legally  an  agent 
of  the  Government  offering  his  best  efforts  for  a fee  --  not  a profit  -- 
to  work  towards  some  goal,  changeable  at  Government  will,  which  he 
estimates  broadly  will  cost  the  Government  some  amount  of  dollars 
to  achieve  using  the  approach  he  intends  to  take.  But  he  is  responsible 
neither  for  achieving  this  goal  nor  for  his  cost  estimates,  "Price" 
thus  does  not  enter  the  picture  at  all. 

Under  the  circumstances,  for  the  Government  to  take 
the  degree  to  which  one  bidder  provides  a lower  estimate  of  cost  than 
another  seriously  enough  to  base  an  award  on  it  is  an  absurdity.  The 
estimate  may  bear  little  or  no  relationship  to  what  the  Government's 
ultimate  cost  for  the  program  will  actually  be  --  whether  because 
the  program  changes  a good  deal  as  it  unfolds  and  the  knowledge  is 

[8] 

This  results  from  a recognition  of  the  uncertainties  in  the  statement 
of  need,  as  well  as  in  performance  and  cost  estimates,  that  ordinarily 
exist  in  such  a highly  complex  and  long-range  activity  as  a major  sys- 
tem program. 
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gained  which  is  the  very  purpose  of  the  development  effort,  or  because 

1 

I the  program's  high  uncertainty  prevents  a valid  cost  estimate  to  be  made 

j of  even  a fixed  program  specification.  This  is  why  "cost  realism",  not 

j estimated  costs,  has  been  the  official  criterion  since  the  '50s  of  judging 

I the  dollars  involved  in  such  programs  --  in  an  attempt  to  penalize 

bidders,  not  reward  them,  for  deliberately  bidding  too  low. 

These  two  "abuses"  thus  lead  themselves  to  direct  attack  — 
through  policy / regulations  backed  up  by  rewards  and  penalties,  education, 
and  research  into  why  the  system  has  drifted  away  from  what  it  knew 
well  in  the  Korean  War  days.  An  attitude  on  the  part  of  Government  pro- 
curement officials,  at  all  levels,  of  questioning  why  an  estimated  figure 
is  so  much  lower  than  other  bidder  estimates  and  independent  Government 
; estimates,  and  negotiations  that  throw  the  burden-of-proof  on  the  bidder  to 

justify  a low  estimate  (it  can't  be  overlooked  that  he  may  in  fact  have  come 

I 

up  with  a better  way)  should  decidedly  help  in  preventing  a "buy-in".  And 
the  same  attitude  of  looking  every  time  for  specifically  supported  cost 
realism  with  regard  to  revision  of  original  estimates,  after  the  necessary 
detailed  discussion  of  original  proposals  has  taken  place,  should  go  a 
long  way  towards  preventing  bidders  from  feeling  that  they  are  in  an  auc- 
tion. Finally,  the  cultural  heritage  that  has  caused  the  blind  worship  of 
the  "low  bid'  for  so  long  --  particularly  in  the  Congress  and  in  its  inves- 
tigating arm,  the  GAO  --  predictably  will  be  tempered  to  a degree  as 
the  legislative  branch  sees  the  continuance  of  the  cost  "over-run"  problem 
for  so  long  and  begins  to  seriously  investigate  why  it  is  happening.  They 
will  quickly  discover  (many  are  already  well  aware)  that  a significant 
portion  of  it  is  caused  by  the  under-estimated  figures  they  get  for  major 
programs,  whose  acceptance  by  DOD  they  decidedly  contribute  to  by 
j their  carry-over  to  such  programs  of  the  cultural  norms  of  other  Gov-  i 

t 

L 
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ernment  procurement.  The  Congress  has  in  fact  already  taken  a 
step  in  the  right  direction,  through  its  proposed  Federal  Acquisition 
Act.  This  Act  recognizes  that  "price"  may  well  not  be  a "primary 
or  significant  factor"  in  an  acquisition  calling  for  competitive  nego- 
tiations. 


(2)  "Technical  leveling"  is  an  abuse  which  it  is  claimed 
Government  proposal  evaluators  indulge  in  by  discussing  all  the 
technical  proposals  with  the  various  competitors,  before  one  of 
them  is  selected,  in  such  a way  that  they  all  end  up  with  essentially 
the  same  technical  proposal  or  proposals  of  relatively  equal  tech- 
nical merit;  and  the  source  selection  can  then  be  made  on  the  basis 
of  the  most  objectively  measureable  and  hence  most  readily  defendable 
basis:  lowest  price^'^^  This  is  not  a defensible  practice  even  in  the 
case  of  a fixed-price  type  contract  relationship,  i.  e. , even  when  the 
notion  of  "price"  i_s  valid.  But  the  solution  to  it  is  not,  again,  to 
reduce  the  opportunity  for  it  to  happen  by  eliminating  areas,  and 
reducing  the  numbers,  of  discussions  of  technical  proposals  that 
take  place,  as  "four-step"  does.  Rather,  it  is  to  face  up  directly 
to  the  fact  that  a broad  menu  of  possible  approaches  among  which  the 


[9] 


In  defense  of  the  motivation  of  Government  evaluators  here,  it 
might  be  observed  that  what  these  evalxxators  seem  to  be  trying 
to  do  in  a complex  source  selection,  consciously  or  unconsciously, 
is  to  help  make  all  the  technical  proposals  with  any  merit  more 
"acceptable"  --  in  what  appears  to  be  in  the  Government's  best 
interest.  However,  the  result  of  their  efforts  is  decidedly  to 
narrow  the  differences  among  the  bidders'  proposals,  to  the  degree 
at  times  that  a conclusion  is  reached  that  there  is  no  technical 
basis  for  selecting  one  bidder  over  another.  Some  other  basis 
must  be  used,  viz.,  price. 
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Government  can  choose  is  in  the  Government's  best  interest,  in 
contrast  to  "leveling;"  or  narrowing  the  Government's  choices  for 
whatever  reason.  This  recognition  is,  in  fact,  a major  tenet  of 
the  new  A- 109  systems  acquisition  policy  now  being  instituted  in 
the  Department  of  Defense  (in  fact,  throughout  the  Government). 
Encouraging  it  and  facing  up  to  the  need  for  getting  into  a position 
to  do  the  admittedly  difficult  job  of  comparatively  evaluating  what 
may  be  quite  diverse  technical  proposals  under  it,  in  a fashion 
that  will  stand  up  to  third-party  scrutiny,  will  have  to  be  accom- 
plished directly  by  DOD  in  any  case.  There  is  therefore  no  need, 
much  less  desirability,  for  additional  DOD  procurement  policy  to 
deal  with  the  problem,  especially  when  such  additional  policy  deals 
with  the  problem  in  a restrictive  rather  than  a positive  way. 


(3)  Finally,  there  is  the  claimed  abuse  known  as  ''technical 
transfusion".  This  is  the  deliberate  taking  of  the  better  parts  of 
losing  proposals  and  making  them  a part  --  or  "transfusing"  them  - - 
with  what  the  winner  is  offering.  This  is  considered  a form  of  stealing 
by  the  losers,  whose  ideas  may  even  involve  proprietary  material. 

As  a minimum,  it  certainly  doesn't  encourage  bidders  to  come  forth 
with  their  best  ideas  in  a source  selection  procedure  until  they  ab- 
solutely have  to. 

But  tlie  answer  to  the  problem  isn't  something  like  "four- 
step",  which  reduces  the  possible  number  of  rounds  and  eliminate'" 
parallel  negotiation  in  a source  selection  and  hence  theoretically  pro- 
vides little  opportunity  for  transfusion  or  holding  back.  On  the  contrary, 
"teclinical  transfusion"  should  be  encouraged.  As  indicated  above,  get- 
ting the  best  possible  response  to  the  Government's  need  is  the  primary 
responsibility  of  the  Government  people  involved.  But  this  encouragement 
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needs  to  be  accompanied  by  a method  for  rewarding  losers  for  those 
of  their  ideas  that  are  used,  i.  e. , a means  of  providing  a positive 
incentive  for  creativity  and  innovation  by  directly  paying  for  it,  and 
paying  for  it  on  the  basis  of  its  value,  not  simply  its  cost. 

This  is  admittedly  not  easy  to  do.  But  one  way  of  doing 
it  already  exists  and  is  practiced  at  times.  And  so  it  can  be  done. 

And  that  is  to  reward  a loser's  good  idea,  when  it  is  a separable  piece 
like  a subsystem  or  an  item  of  equipment,  to  the  same  degree  the 
winner's  ideas  are  by  directing  a subcontract  to  the  loser  for  the  piece 
throughout  the  program. 

Where  it  is  not  a separable  piece,  a greater  challenge 
seems  to  be  present.  However,  even  here  there  would  not  appear  to 
be  any  reason  why  something  like  a royalty  over-ride  for  desired 
technical  services  by  the  loser  could  not  be  provided  for  in  the  winning 
contract,  in  an  amount  negotiated  by  the  Government  with  the  loser 
sufficiently  above  the  out-of-pocket  costs  of  providing  the  technical 
services  to  reflect  the  value  of  the  idea  being  used.  An  arrangement 
similar  to  this  was,  in  fact,  consummated  by  the  Navy  with  a number 
of  suppliers  during  the  Korean  War  when  multiple  sources  were  needed 
in  a hurry  on  various  items. 

In  sum,  if  increased  equity  is  the  goal  of  "four-step", 
why  not  provide  for  it  directly  in  a way  that  is  fundamental  to  our 
Society:  through  respect  for  property  rights,  whether  thpse  rights 
are  tangible  or  not? 
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CONCLUDING  COMMENT 

To  the  degree  that  the  new  Government -wide  process  for 
acquiring  major  systems  that  is  mandated  by  OMB  Circular  A- 109 
of  April  1976  is  implemented  by  parallel  developments  and  other 
"hardware"  type  continuous  competitions  of  results,  rather  than  by 
ad  hoc,  winner -take -all  "paper"  competitions  of  promises,  some  of 
the  above  concerns  disappear  to  a degree.  Losers  will  be  paid  some- 
thing for  their  ideas,  albeit  not  necessarily  their  true  value;  selec- 
tions will  be  based  more  on  things  that  can  be  measured,  thus  both 
reducing  the  importance  of  exploratory  discussions  in  source  selec- 
tions and  the  opportunity  for  technical  leveling;  and,  finally,  costs 
will  be  able  to  be  estimated  much  more  realistically  by  all  parties, 
reducing  the  likelihood  of  both  "auctioning"  and  "buying  in".  However, 
to  the  very  degree  that  A-109  reduces  these  concerns  with  "four-step", 
it  raises  the  direct  question  of  why  "four-step"  is  needed  at  all  in  the 
new  milieu.  At  the  very  least,  does  it  not  suggest  that  "four-step" 
ought  to  be  confined  to  only  those  competitions  in  which  a single  source 
is  to  be  selected  as  a result  of  paper  proposals,  and  where  program 
results  are  sufficiently  certain  that  the  contracting  relationship  can  be 
of  the  fixed-price  type?  And  even  in  these  cases,  the  difficulty  "four- 
step"  gives  bidders  in  trying  to  satisfy,  through  tradeoff  analyses, 
other  current  acquisition  policy  objectives  of  DOD  such  as  design-to- 
cost  and  cost  estimating  on  a life -cycle  cost  basis^  ^^^akes  it  of 
questionable  value.  Should  it  thus  be  considered  at  all  at  this  time? 

lio]  ~ 

See  DOD  Directive  4105.62  of  January  6,  1976,  "Selection  of 
Contractual  Sources  for  Major  Defense  Systems,"  for  various 
new  acquisition  policy  objectives. 
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Pentagon  Plans  New  Contracting  Proems 

To  Cut  Cost  Cherpins,  Push  Innovation 

-- 

Hy  Kennbth  H.  ti\c<fs/  ' 
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Corporations  gener.clly  suppoit  the  new 
piocedure,  wh-rh  the  Pentagon  plans  to 
1 dupt  in  .several  months  (or  most  contracts 
n design.  devePsp  and  huild  major  weapons. 

('  'irenlly,  when  companies  bid  on  a proj 
C(...  each  as  a iielicopter,  they  submit  com- 
plet  * proposids  including  design  plans  and  a 
puC'',  After  evaluating  the  proposals,  the 
■lenice  ordering  the  weapon  begins  a round 
< ; separate  negotiations  with  .ill  the  bidders 
during  which  it  discusses  the  technical  defi- 
ciencies of  each  proposal. 

Then  companies  are  permitted  to  submit 
a "best  and  final  offer."  Sometimes  there 
are  several  rounds  of  those  offers  before  the 
service  selects  a winner.  If  a weapons  pro- 
gram Is  just  beginning.  Ihe  service  may  pick 
two  or  more  companies  to  design  and  build 
prototypes.  Later  it  will  award  one  company 


cade  oi  more  and  cost  billions  of  dollars. 

Tne  current  systeiqy.haifrii';  to  enc  lurage 
accepting  the  !nwost(price],*  rather  than  the 
best  technical  propo^ff^cording  to  Dale 
Church,  deputy  under  Secretary  of  Di  fense 
for  acquisition  policy.  In  addition,  hi  says 
the  present  procedure,  which  has  been  used 
for  about  20  years,  leads  to  other  abiii.fs. 

One  abuse,  called  "technical  levihng" 
tends  to  obscure  de.sign  differences  boiween 
[iroposals  and  sometimes  encourages  com- 
panies to  cut  comers  on  their  initial  techni- 
cal proposals.  This  is  because  the  discussion 
of  design  deficiencies  gives  companies  a 
chance  to  learn  if  competitors  have  come  up 
with  better  ideas  and  then  to  incorporate 
them  in  their  own  final  proposals.  "It  gives 
an  unfair  advantage  to  the  fellow  who  comes 
in  with  an  inferior  propo.sa!,"  Mr,  Babione 
says. 

Another  abuse  is  called  "auctioneering" 
which  occurs  when  services  require  a series 
of  "liest  and  final  offers,"  each  allowing -or 
forcing,  some  contraclor.s  say- bidders  to 
keep  lowering  their  ^cesf 

"As  a result,  v/e~ofTen  c-t  a price  that 
was  sadly  unrealistic,"  Mr.  Church  says. 
"Using  the  auctioneering  technique  almost 
always  buys  you  problems.  There's  a tre- 
mendous amount  of  inefficiency  created  in 
trying  to  meet  unrealistic  cost  goals." 

The  Four  Steps 

In  contrast,  the  foiir-steo  process  <;  de- 
signed to  minimize  such  horsetradiiig  The 

-First,  submission  of  technical  proposals 
without  detailed  cost  data.  The  service  will 
ask  questions  to  clarify  the  proposals,  but. 
to  prevent  technical  leveling,  major  design 
or  management  deficiencies  won't  be  dis- 
cussed with  bidders.  "This  will  encourage 
all  contractors  to  go  out  and  really  compete 
hard  to  come  up  with  the  best  technical  pro- 
posal the  first  time  around,"  Mr.  Church 
says. 

-Second,  submission  of  cost  proposals. 
These  will  be  discussed  and  evaluated  to  de- 
termine if  the  costs  are  realistic.  By  requir- 
ing companies  to  submit  separate  design 
and  cost  pioposals,  officials  hope  the  bid- 
ders will  put  more  thought  into  each  one. 

-Third,  setting  a common  cut-off  date 
for  (TnaT proposals,  with  a strict  prohibition 
against  asking  for  new  "final"  offers  unless 
the  service  makes  major  changes  in  its 
specifications.  The  service  will  then  select 
the  best  prop'sel 

-Fourth,  aegotiating  a contract  with  the 
vnnner.  For  programs  involving  competitive 
design  or  prototype  development,  contracts 
will  be  award-'sl  to  more  than  one  company. 

"There  simply  aren't  any  panaceas  in 
this  business,"  Mr  Church  cautions.  "Mis- 
takes happen  " But  he  thinks  Ihe  new  pi-oce- 
diire  promises  to  reduce  nsks  for  both  con- 
tractors and  the  government. 


